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ABSTRACT 

This report documents the evaluation of the Bilingual 
Mathematics and f rrience Achievement Program (Project BMSA) for 
students of limij id English proficiency. The bilingual program was 
designed to provide intensive mathematics and science instruction, 
using mastery level concepts, in the native language and to 
incorporate mathematics and science concepts smd themes into English 
language classes. The project provided training for teachers of 
stvdents of limited English proficiency by offering workshops in 
science and mathematics and developed curriculum materials for both 
subjects. To evaluate the program, OREA interviewed project 
coordinators, observed a science workshop, anc. collected data, on 
participants' ratings of the workshops. This document summarizes the 
results of an evaluation of the staff development aspect of the 
program. The peer teaching aspect of the program (Project SUMA) is 
br-r^fly mentioned. A brief overview and history f the program, a 
sununary of the evaluation methodology, and e. summary of the 
evaluation findings of the evaluation of the mathematics and science 
comonents are presented. (KR) 
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BILINGUAL MATHEMATICS AND SCIENCE ACHIEVEMENT 
(PROJECT B.M.S.AO 
1988-89 

SUMMARY 



• The Bilingual Mathematics and Science Achievement 
project was fully implemented. During the 1988-89 
school year, the project provided staff development 
activities in mathematics and science for bilingual 
teachers. It also created science centers, implemented 
a peer tutoring prograun in mathematics, and developed 
curriculxun materials for both mathematics and science. 

• The project met the objectives in staff development for 
both mathematics and science and the peer tutoring 

obj ective. 

The New York City Board of Education funded the Bilingual 
Mathematics and Science Achievement progrcun (Project B.M.S.A.) 
for its first year of implementation. The project provided 
training for over 900 teachers of students of limited English 
pjuoficiency (LEP) by offering workshops in science and 
mathematics and developed cuniculum materials for both subjects. 
Its mathematics component had a peer tutoring program, on which 
the Office of Research, Evaluation, and Assessment (OREA) has 
prepared a separate evaluation report. It also developed 
Bilingual Science Technical Assistance Centers (BI-STACs) . 

To evaluate the program, OREA interviewed project 
coordinators, observed a science workshop, and collected data on 
participants* ratings of the workshops. The program achieved the 
staff development and peer tutoring objectives. 

The positive aspects of the program included the hands-on 
approach to teaching science and mathematics, the promising 
features of the peer tutoring program, and the practical value of 
the BI-STACs. 



ERIC 



5 



TABLE OF CONTENTS 



I. INTRODUCTION 1 

History of the Program 1 

Setting 2 

Participants 2 

Staff 2 

Delivery of Services 3 

Report Format 3 

II. EVALUATION METHODOLOGY 4 

Evaluation Questions 4 

Evaluation Procedures 4 

Sample 4 

Instruments 4 

Data Collection and Analysis 5 

III. EVALUATION FINDINGS 6 

Mathematics Component 6 

Peer Tutoring • 6 

Staff Development 6 

Science Component 8 

Bilingual Science Technical Assistance 

Centers Staff Development 9 

Elementary Science Program Evaluation 

Test Staff Development 11 

Curriculum Development 11 

Needs Assessment Survey 11 

IV. CONCLUSIONS 12 



ERIC 



ii 

6 



I . INTRODUCTION 



This report documents the Office of Research, Evaluation, 
and Assessment's (OREA's) evaluation of the New York City Board 
of Education- funded program. Bilingual Mathematics and Science 
Achievement (Project B.M.S.A.) for students of limited English 
proficiency (LEP students). The project, administered by the 
Division of Multilingual/Multicultural Education (DOMME) of the 
New York City Board of Education, completed its first year of 
operation. Its primary goal was to provide intensive mathematics 
and science instruction, using mastery level concepts, in the 
native language and to incorporate mathematics and science 
concepts and themes into English language classes. 

One of the components of Project B.M.S.A. was peer rutoring. 
Under the title of Students Upgrading Mathematical Achievement 
(Project SUMA) , this component was the object of a separate 
evaluation by OREA. Project SUMA's primary objective was to 
provide remedial tutoring to students lacking matiiematical skills 
and enrichment to students who were potentially gifted in 
mathematics. 

HISTORY OF THE PROGRAM 

Growing evidence has indicated that students at all levels, 
particularly LEP students, fail to acquire basic competency in 
mathematics and science. According to official statistics, 
language-minority students of all national backgrounds frequently 
fail to pass the Regents Competency Tests (R.C.T.s) in 
mathematics and the sciences. Research studies have indicated 
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that teachers, particularly at the elementary and junior high 
school levels, who increase their competency in science and 
mathematics, overcome their negative feelings towards these 
subjects. As a result, they improve their teaching skills and 
their students' learning. 

Science centers were established in Manhattan, the Bronx, 
and Brooklyn. Staff development workshops took place in the 
Manhattan, the Bronx, Brooklyn, and Queens. 

PARTICIPANTS 

Some 870 teachers from all over the city participated in the 
staff development component of the project. The project highly 
recommended participation in the mathematics workshops to new 
bilingual teachers, but encouraged participation of all teachers 
of LEP students. 

STAFF 

The proposal called for a bilingual supervisor, four 
teachers, one office aide, and four teacher aides. The bilingual 
supervisor was to act as the director of the mathematics 
component, but he was given other responsibilities outside the 
pro'ject and the project supervisor assumed his duties. The 
project supervisor was responsibile for the overall coordination 
of the project, the recruitmer.t of participants, and the hiring 
of consultants. One of the four teachers acted as the 
coordinator of the science component. Her duties included 
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setting up the science centers, conducting staff development 
activities, and providing support services* A bilingual 
paraprofessional served at each of the centers. 

DELIVERY OF SERVICES 

The program provided training in mathematics and science, 
using lectures, hands-on activities, discussions, and topical 
written materials. 

REPORT FORMAT 

Th-'s report is organized as follows: Chapter II describes 
the evaluation methodology; Chapter III presents an analysis of 
the findings of the evaluation; Chapter IV offers conclusions 
based upon the results of the evaluation. 



II. EVALUATION METHODOLOGY 



EVALUATION QUESTIONS 

The evaluation assessed prograiQ implementation. Evaluation 
questions included the following: 

• Were Bilingual Science Technical Assistance Centers 
(BI-STACS) established as planned? 

• Did the program provide workshops at the BI-STACS? 

• Did the program conduct training on the Elementary 
Science Program Evaluation Test (ESPET)? 

Did the program provide a staff development program in 
mathematics? 

• Did the program review and evaluate bilingual 
materials? 

• Did the program design bilingual materials in science 
and mathematics? 

EVALUATION PROCEDURES 
Sample 

An OREA field consultant interviewed the project supervisor, 
the science coordinator, and the former matiiematics supervisor. 
She also observed a science workshop at the BI-STAC in P.S. 128 
in Manhattan. The project returr^ed 351 completed mathematics 
workshops* evaluation surveys in the fall and 554 in the spring. 
Instruments 

OREA developed interview and observation schedules. Project 
personnel used an OREA--developed survey to assess participants' 
evaluations of workshops they had attended. The survey consisted 
of a four-point Likert-type rating scale of eight statements 
about objectives, organization, j ob-relatedness , the presenter, 
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the materials, and an overall assessment. Possible answers 
ranged from excellent to "poor.** Surveys also included three 
open-ended questions which qpaeried participants about the 
strongest feature of the workshop, suggestions for future 
workshops, and comments. 

Data Coll ection and Analysis 

Interviews were conducted and observations made during the 
month of March. OREA collected the evaluation surveys at the end 
of the school year. Data analysts performed frequency 
distributions of each of the Likert-type items. 
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III. EVALUATION FINDINGS 

MATHEMATICS COMPONENT 

The mathematics component included peer tutoring and staff 
development. 

Peer Tutoring 

A final evaluation report on the peer tutoring component, 
Project SUMA, gives a detailed description of its av^^tivities 
The project met the objective that it would establish the 
bilingual peer tutoring program in at least two schools. 

staff De velQPmenr 

The program objective for staff development was: 

• To provide ovar 500 teachers with a staff development 
program in the area of mathematics. 

The project designed the staff development component to 

assist iJilingual, English as a Second Language (E.S.L.), and 

monolingual teachers of LEP students with mathematics 

instruction. The project offered four series of workshops, two in 

the fall and two in the spring semester. Each series consisted 

of four sessions, each session was three hours lor.g. Topics 

included teaching mathematics problem-solving, instructional use 

of computers, teachinct mathematics through language arts, using 

manipulatives in the teaching of mathematics, integrating i:.S.L. 

and mathematics instruction, hands-*on teaching of mathematics, 

methodology in the teaching of mathematics, state requirements 

for mathematics, PEP test and syllabus, understanding children's 
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mathematical thinking (the roles of language and culture, context 
connection), mathematics and language, meaningful learning, and 
practical mathematics. 

Fall workshops took place during October, November, and 
December at Aviation High School in Queens and Brooklyn Technical 
High School. Jpring workshops took place during February, March, 
and April at Martin Luther King, Jr., High School in Manhattan 
and at P.S. 105 in the Bronx. 

Limitations to the effectiveness of this kind of training 
included the difficulty of having to attend a three-hour session 
that required as ?nuch as one hour of travel time after a full 
workday. 

Workshop Evaluation . OREA evaluated fall and spring 
workshops separately. Project B.M.S.A returned 313 completed 
evaluation surveys in the fall. Judgments were overwhelmingly 
positive. Sixty-nine percent of the participants gave the 
workshops an overall ra*:lLg of excellent, 26 percent rated them 
as good, and less than three percent considered them fair or 
poor. At least 9b percent of the participants gave a rating of 
excellent or good to the following items: clarity of objectives, 
achievement of objectives, organization of the workshop, 
presenter's knowledge, and presenter's effectiveness. 

Common answers among the 108 teachers who identified the 
most important feature of the workshops were their practical use 
(28 percent), the materials/handouts {25 percent), and the 
approach to the topic (19 percent). Fifty-three percent of 
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t:eachers offering recommendations suggested more workshops; 21 
percent wanted more material!*. 

The project submitted 554 completed evaluation surveys in 
the spring. Sixty-five percent considered that the workshops 
deserved an overall rating of excellent, 24 percent assessed them 
as good, and only five percent considered them fair or poor. A 
lainimtim of 95 percent of the participants rared the following 
categories as being either excellent or good: clarity of 
objectives, organization of the workshop, and presenter's 
knowledge . 

Project B.M.S.A. served over 867 participants citywide, well 
abavB the intended 500. It met the mathematics staff development 
objective. 

SCIENCE COMPONENT 

The science component included the creation of BI-STACS, 
staff development, and the initiation of the Bi-Tech Science/Math 
newsletter. The newsletter disseminated information on the 
latest educational developments in science and mathematics. 

The project proposed to develop problem-solving and inquiry 
skills for this component using a hands-on approach. The 
workshops pursued two basic objectives: to improve science 
instruction in the native language and in E.S.L., and to conduct 
training in the Elementary Science Program Evaluation Test 
(ESPET) , a newly implemented test for fourth grade students. 
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Bilingual Science Technical Assistance Centers Staff Dev elopment 
The program objective for BI-STACs was: 

• To provide training to 60-80 teachers at each of the 
Bilingual Technical Assistance Centers (BI-STACS) , to 
improve science instruction in the native language and 
English as a Second Language (E.S.L.)* 

BI-STACS were resource centers for teachers and were 
furnished with books, journals, and various materials and 
leUDoratory equipment for science and mathematics experiments. 
They provi'led technical assistance, encouraged the use of science 
and mathematics, and trained teachers to facilitate the learning 
of these subjects by making students aware of their u*=^e in day- 
to-day living. A paraprofessional staffed each center, providing 
orientation a id performing other tasks. 

The project opened and fully implemented a BI-STAC at P.S. 
128 in Manhattan and one at I.S. 184 in the Bronx. Because of 
structural problems at I.S. 291 in Brooklyn, the planned BI-STAC 
was unable to open during the year under review. Similarly, the 
project cou^d not open the BI--STAC in Queens. 

The utilization statistics for the two fully-operating 
BI-STACs show that they served a total of 1,859 people, 
distributed in the following way: 1,139 elementary and junior 
high school teachers, 166 parents, and over 500 teachers 
visitors. Project staff provided workshops designed to improve 
science instruction in the native language. Topics included: 
» how to encourage science and mathematics learning at home, 

problem solving, and consumer behavior, among others. The 
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science coordinator also provided workshops at schools by 
request • 

An OREA field consultant attended one of the science 
workshops F.S. 128. There were 14 participants from four 
Commtinity Schoo^ Districts (C.S.D.s) present- In an activity on 
one-way coxanunication, a single participant followed verbal 
instructions on how to set up some colored geometric figuras. 
This participant covld neither ask for an explanation or a 
reformulation of the instructions but only say "repeat." After 
th^ experiment, the instructors made an analysis of the multiple 
fc^,nefits students could derive from an experience such as this: 
the learning of shapes, colors, and concepts swch as "left," 
"upside down," "reverse," and "oppor,ite; " the w-nderstandinj of 
cooperation and of listening skills; an end to the perception of 
science as a reading lesson; and, finally, the use of science as 
motivation for reading, writing, mathematics, and art. Project 
staff distributed handouts with j^uggestions for additional in- 
class activities. 

At the end of the workshop, participants offered oral 
feedback to the presenters. All reedback was positive and dealt 
with the materials used ("I can use things that I have 
available"), the approach ("Children will not even think that 
this is science," and "This is too much fun to be science"), or 
with the need for further training of this kind ("What we really 
need is more workshops like thir> one"). The project met the BI- 
STAC objective. 
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Elementary Science Program Evaluation Test Staff Development 
The objective for staff development was: 

• To conduct training on the Elementary Science Program 
Evaluation Test (ESPET) . 

The project director actively participated in regional 

training wor^^shops on the administration and scoring of the 

Elementary Science Program Evaluation Test (ESPET) . Science 

teachers were particularly enthusiastic about this, because they 

felt th'-»t it would correct weak areas and ensure their ability to 

provide students with quality experiences in elementary science. 

Project RT-t;TAC achieved the objective in training on the ESPET. 

CURRICULUM DEVEL0FI4ENT 

The project undertook several activities in the area of 
curriculum development. It identified, reviewed, and listed 
bilingual materials; evaluated materials developed by the 
C.S.D.s; and designed bilingual materials (lesson plans, syllabi, 
glossaries, course outlines) for use by trainers and teachers. 

NEEDS ASSESSMENT SURVE Y 

Staff members felt that the workshops would be more useful 
if they responded to teachers* needs. Therefore, B.M.S.A, 
designed a checklist on topics a teacher should know about 
mathematics and science and sent over 700 questionnaires to be 
completed by teachers at several C.S.D.s. About 80 percent of 
the questionnaires were returned. The project intends to plan 
next year's workshops on the basis of these questionnaires. 
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IV. CONCLUSIONS 

In its first year of operation, the Bilingual Mathematics 
and Science Achievement program achieved a high level of success 
in implementing its primary goal: to provide intensive 
mathematics and science instruction to teachers. 

Project B.M.S.A. met the staff development objectives for 
its two components, as well as the peer tutoring objective for 
the mathematics component. It also implemented and made two 
Bilingual Technical Assistance Centers fully operational. These 
were designed to serve as resource centers for teachers. 

Project B.M.S.A. was successful in providing workshops that 
enhanced the understanding of science and mathematics among 
teachers of LEP students. Participants rated the workshops 
positively and considered their practicality to be one of their 
most important features. 

The positive aspects of the program included the hands-on 
approach to teaching science and mathematics, the promising 
features of the peer tutoring program, and the practical value of 
the BI-STACs. 



ERiC 



12 



18 



Appendix 16 



ERIC 



END 

U.S. Dept. of Education 

Office of Education 

Research and 
Improvement (OERI ) . 

ERIC 



Date Filmed 
March 29, 1991 



